Experience in Testing Compiler Optimizers Using Comparison Checking

This paper describes our experience of testing and debugging an optimizer using comparison checking. Although this study is based on Jaramillo et al.’s work, the experience will help those who test optimizers using this technique. In our implementation, important values during the execution of programs are output as a file trace before and after each optimization. Then a comparison phase checks these results. When the comparison checker finds an error in the optimizer the system shows a C language style program that is back translated from the intermediate code. Therefore, the optimizer writer can easily find the erroneous section of the optimizer. We have implemented the system on a compiler infrastructure and have verified the optimizers that our group has been developing. By applying this technique, we found four bugs, including two unknown bugs, in the optimizer.